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Abstract 


An experimental study was conducted to determine the effects 
on human performance for several telemanipulation tasks. Experiments 
subtended visual angles, with and without force feedback. 


of various forms of visual and force feedback 
were conducted with varying frame rates and 


1. Introduction 

In Section two of this paper we describe our research objectives for conducting our experiments that focused 
on human/machine interaction for space teleoperation. 

Section three contains descriptions of the experimental equipment. Our experimental design is discussed in 
Section four. Six test subjects used a master/slave manipulator during two experimental sessions. In one session the 
subjects performed the tasks with direct vision, with and without force feedback, and with the manipulator at three 
different distances from the task board yielding three subtended visual angles: 3.28, 1.64 and 1.09 degrees 
respectively. During the other session, the tasks were performed using a video monitor for visual feedback, with and 
without force feedback, and with three different frame rates (3, 5, and 30 frames/second) for die video transmission 
The tasks were three peg-in-hole type tasks corresponding to 4, 5, and 6 bits/task according to Fitts Index of 
Difficulty. 

Experimental results for the video viewing and direct viewing environments are presented in Section five. 
The experimental data were analyzed through an analysis of variance. The video viewing results showed that frame 
rate, force feedback, task difficulty, and the interaction of frame rate and force feedback made significant differences in 
task times. For the direct viewing environment, subtended visual angle, force feedback, task difficulty, an e 
interaction of subtended visual angle and force feedback made significant differences in task times. Also in Section 
five are the results of comparing performance between the video and direct viewing environments. Comparable visual 
feedback to the human operator was provided by (a) the 1.64 subtended visual angle for direct viewing, and (b) thirty 
frames/second frame rate for video viewing. This allowed for an analysis between the direct and video viewing 
environments. While force feedback and task difficulty made significant differences in task times, the view itseli 
(video vs. direct) did not. 

Conclusions and suggestions are made in Section six based on the research results to help facilitate improved 
teleoperator performance for space operations. 

Additional information on this experimental study can be found in [1]. 


2. Research Objectives 

Performing a task directly with hands and eyes, unimpeded by physical distance, hardware, or artificial 
communication, has been observed experimentally to be quicker, easier, and usually more accurate than performing a 
task remotely using video and a remote manipulator device [2], However, not all space tasks are well suited to be 
done manually by astronauts in extra-vehicular activity (EVA) due to the nature of the task, the risks and nature of the 
space environment, or the astronaut being constrained by his/her gloves or pressure suit. Thus there are safety, cost, 

and efficiency considerations that can make EVA procedures undesirable. Many researchers and engineers, ac wit 
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the necessity of remote viewing and handling, have sought to understand how human/machine interaction for the 
control of space teleoperators can be improved [3-5]. Nevertheless, knowing when to have manipulation tasks done 
by astronauts in EVA, remote manipulators controlled by humans, or autonomous robots is an important current 
issue. 


The human operators performance in telemanipulation can surely be improved by providing the operator 
with appropriate and adequate feedback. Teleoperation will not eliminate or drastically alter the need for humans in 
space, but it will alter the roles that humans fulfill in space and should improve human productivity there. The use 
of remote manipulators should also free up valuable crew time to be spent on other space operations and experiments. 


The elements of a remote viewing and manipulation system include a video camera, a telecommunication 
channel, a video display, human eyes, human arms and hands, a master arm, a slave or robot arm, and the task 
operation itself (such as putting a peg into a hole). These elements have not only desirable but also undesirable 
properties. They tend to add undesirable forces, displacements,, time, illumination, and contrast to the remote 
environment that would not be there in the manual situation. They can be perceived as "filters" that prevent 
information from reaching the human operator, and thus retard performance [2]. In the experiments presented in this 
paper the effects of various degrees of some of these "filters" were investigated including: 

* Force feedback and its effects on motor capabilities. 

* Video frame rate, and its effects at different values on performance. 

* Subtended visual angle and its effects on task performance for manipulation with 
direct viewing. 

* Task difficulty and its effects on the cognitive and motor capabilities of human 
operators. 

* The use of a video monitor versus the use of direct vision for remote manipulation. 

* The interactions of two or more of the above variables with each other and the 
corresponding effects on performance. 

These filters and their effects need to be identified and quantified to provide information to facilitate 
efficient teleoperation in space. Experimental findings, quantified and analyzed through statistical methods, should 
prove helpful to researchers and policy makers alike. The results could be implemented over a variety of space 
applications including: 

* space shuttle remote manipulator system [6] 

* control of an orbital maneuvering vehicle with a robotic front end [7] 

* teleroboiic servicer on the space station 

* a number of planetary and lunar missions such as a Mars sample return mission or 
lunar exploratory operations [8-9]. 

The major goal of the experiments presented here was to provide information to help technologists better 
understand what the capabilities of humans are when interfacing with space telemanipulators under various sensory 
feedback conditions. 

3. Experimental Equipment 

E2 Manipulator System 

The E2 master-slave manipulator had the capability of operating with direct electronic coupling control both 
bilaterally with force feedback, and unilaterally without force feedback coming from the slave back to the master arm. 
Our E2 was right handed, as were all of the test subjects used in the experiments. Both master and slave arms had 
seven degrees of freedom including end effector gripping, were geometrically similar, and were kinematically 
isomorphic to the operator's arm and hand. The manipulator degrees of freedom are shown in figure 1. The E2 had 
seven degrees of freedom: three (x, y, and z direction) for arm translation, one for arm rotation (azimuth), one for 
gripper elevation, one for gripper twist, and one for the grasping motion of the gripper jaw. Tracking time delays 
were considered negligible during the experiments due to the manipulator's quick and accurate response to the 
operator’s input motions [10]. 
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Figure 1 - E2 manipulator degrees of freedom. Source: [10] 

Video System 

Central to altering the video environment was the capability of varying the J:rame ra ^- This ^ 
accomplished through the use of the AT & T Truevision Advanced Raster Graphics Adapter (TARGA 16) board and a 
computer program. The resolution selected for the experiments was 512 X 256 pixels in order to have non-interbed 
video input or output. TARGA captures images in real time:l/60th of a second per field or l^Oth per tame. Once a 
frame rate was selected, the TARGA board would capture a frame at the rate necessary to provide the requested frame 
rate. A color monitor provided the visual feedback to the test subjects. 


Task Board 


The task board consisted of four slots. Each slot was made with two side boards and a center board that 
formed a back. Figure 2 displays the task board dimensions. The two end slots were 3.75 inches wide and the middle 
slots were 3.25 and 3 inches wide respectively. The block used in the experiments was 2.75 inches wide, providing 
task tolerances from left to right of one inch, one-half inch, one-quarter inch, and one-inch. The redundancy r of .the 
one inch tolerance on the outer slots allowed the alternating of right and left mouon. The centers of each of the ou 
slots were eight inches apart. Mounted on the lower portion of each center block were limit switches that controlled 

the clock to record the task times. 
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Figure 2 - Task Board Layout 
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4. Experimental Design 

Tasks 


The tasks consisted of moving the block of wood on the task board with the manipulator arm. Thus the 
distance moved (eight inches) versus the tolerance of fit (1,.5,.25 inches) provided multiples of 2 for easy use of Fitts’ 
law [1 1] which applies the formula: Index of Difficulty (Id) in bits per task equaled log2(2A/B), where A equaled the 
distance moved and B equaled the tolerance of fit. Fitts' law produced indices of difficulty of 4, 5, and 6 bits per task 


Video Vie wing Experiments 


Since communication channels for teleoperation in space are often constrained by limited bandwidth for 
information transmission, decreasing the amount of information that needs to be transmitted can save time and 
money. Decreasing frame rate is one way to decrease the amount of information that is transmitted, and this section 
ot the experiment was designed to measure the effects of varying frame rate on operator performance. The goal was to 

gain information on acceptable frame rates for controlling a remote teleoperator under different face feedback and task 
difficulty conditions. 


T hC c ^ rect b°ard was cut-off from the subject and the video monitor was used with three 

different frame rates: three frames per second (fps), five fjps, and thirty fps, with and without force feedback, for a total 
of six experimental conditions. Thirty frames per second provided satisfactory image fusion, i.e. output that appeared 
as steady motion. Five fps and three fps were selected based on previous experiments and preliminary 
experimentation. Ranadive [12] found that a threshold frame rate existed at three fps beyond which task performance 
was virtually impossible. He also discovered that frame rates below 5.6 fps considerably degraded performance and 
increased variability. Preliminary experimentation confirmed these trends. After trying many frame rates, 30, 5 and 
J tps were chosen since they appeared to represent breaks in the performance curve. 

Direct Vie wing Experiments 


r f U e 3180 P erfbrmed with subjects using direct vision for visual feedback, with and without 

torce feedback. The master manipulator was placed at three different viewing distances from the task board: four 
eight, and twelve feet. Each distance from the task board in the direct viewing experiments had an associated 
subtended visual angle. The formula: subtended visual angle = (57.3) (60) L/D degrees, gives a measure for subtended 
)[ ls ~ x a f g ^ e l where L - 1 * c size of object measured perpendicular to the line of sight, and D = the distance from 
the front of the eye to the object [13]. In this case L = 2.75 inches and D varied between 4 feet (48 inches), 8 feet (96 
inches), and 12 feet (144 inches), yielding subtended visual angles of 3.28 degrees, 1.64 degrees, and 1.09 degrees 
respectively. This allowed analysis of the effects of varying subtended visual angles on task times. 

Direct Viewing at 8 feet an d Video Viewing at 30 fas 


The eight foot distance yielded the same subtended visual angle as the experiments that were performed with 
die video monitor (1 64 degrees). This allowed for a comparison of the 1.64 degrees direct viewing data with the 30 
lps video data since 30 fps appeared to the eye as a steady motion. Thus 1.64 degrees direct data and 30 fps video data 
each had subtended visual angles and frame rates that appeared equal to the human operator. Some of the major 
dmerences were stereo vision with direct viewing, the ability to move one's head to change the viewing line of sight 
with direct viewing, and the different environments surrounding the task board for each view. It was the effects on 

performance of the stereo vision and other differences in the views that were of interest in this section of the 
experimental design. 

Experimental Procedures 


Six MIT graduate students were used as test subjects. Each subject attended a one and a half to two hour 
training session a few days prior to performing the final experimental runs. The subjects were made familiar with the 
experimental design and procedures and became acquainted with thfc E2 manipulator system. They performed each 
task condition and repeated the tasks until they said they were familiar and felt comfortable with the manipulator and 
the manipulation environment. Then they performed the tasks for time just as they would in the actual experiments. 
When their performance times met minimum training levels and the learning effects subsided, the subjects were then 
trained on the next experimental condition. 
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Subjects then underwent two separate experimental sessions: one using the video monitor and the other for 
performing the tasks with direct vision. A balanced latin square was used so that each experimental condition 
preceded and followed every other condition an equal number of times to counterbalance the effects of fatigue and 
learning on the experimental results. 

The video portion consisted of three frame rates, two forms of force feedback, and three different tolerances 
which corresponded to three different indices of difficulty for a 3x2x3 design. The direct vision experiments included 
three visual angles, and the previously stated feedback and tolerance/difficulty parameters for a 3x2x3 design. Eac 
task was performed five times going to the right and five times going to the left for 5x2 =10 tasks per condition. Six 
subjects were used so the video experiments had a total of 3x2x3x5x2x6 = 1080 number of data points, and the direct 
vision experiments also yielded a total of 3x2x3x5x2x6 = 1080 number of data points. 


5. Experimental Results 


The experimental data was analyzed through an analysis of variance (ANOVA) with a 95 percent confidence 
level as described in [14-15], and Newman-Keuls post-hoc testing as outlined in [16], The results of the statistical 
analysis are illustrated in tables 1 and 2 and figures 3 to 8. 


Table 1 - ANOVA results for variables and interactions with statistical significance 
during video viewing performancg 


MU li :7*i 


Frame rate 

2, 10 

Force feedback 

1.5 

Task difficulty 

2, 10 

Interaction of frame rate & force feedback 2, 10 


F-VALUE 

73.1 

167 

15.3 

8.99 


P-VALUE 

0.0001 

0.0001 

0.0009 

0.0058 


Table 2 

- ANOVA results for variables and interactions with statistical significance 
during direct viewing performance 


SOURCE 

Degrees of Freedom 

F-VALUE 

P-VALUE 

Subtended visual angle 2, 10 

Force feedback 1,5 

Task difficulty 2, 10 

Interaction of subL vis. ang. & force feedbk 2, 10 

57.8 

49.5 

27.3 

6.91 

0.0001 

0.0009 

0.0001 

0.013 


Effects of Force Feedback 

As displayed in tables 1 and 2, force feedback made a significant difference in performance. For video 
performance, force feedback was significantly better than no force feedback. With force feedback the average task time 
for all frame rates combined was 2.98 seconds and the absence of force feedback produced a combined average task 
time of 5.29 seconds. 


Even when viewing was direct (no video), force feedback made a significant improvement in the mean task 
times. The presence of force feedback yielded a total mean task time of 1.71 seconds and without force feedback the 
total mean task time was 2.80 seconds. 

Effects of Task Difficulty 


For the video viewing experiments, task difficulty made a significant difference as shown in table 1. The 
Newman-Keuls analysis determined that only the quarter inch tolerance or the most difficult task (Id=6 bits) had 
significantly different mean task times from the other two tasks. The one inch tolerance task with Id=4 and the half 
inch tolerance task with Id=5 were not significantly different from each other. For Id=4 the mean task time was 3.55 
seconds, for Id=5 the mean task time was 3.95 seconds, and for Id=6 mean task time was 4.90 seconds. 

These relationships are more clearly represented in figure 3. The mean task time values are plotted for the 
three different indices of difficulty. The error bars represent the standard error for each mean. A linear regression 
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yielded y-0.758+ 0.675x. An exponential fit was also done, yielding a relationship of time (y) to index of difficulty 
u Pattern of the data points and the results of the Newman-Keuls post-hoc test suggest 

that the exponential plot described the behavior or trends of the data better than the linear plot These were different 
from the linear results that Fitts [11] obtained but were similar to the results of Hill [17-18]. 

Although these results are based on only three data points, each data point is the mean of three hundred and 
sixty data observations. Further, the plots are only meant to indicate general trends in performance. Therefore it is 
with some confidence that these results are presented. 



0123456789 
INDEX OF DIFFICULTY (BITS/TASK) 

Figure 3 - Task Difficulty Effects on Video Viewing Performance 


Table 2 displays that task difficulty also made a significant difference in performance for the direct viewing 
experiments. Post-hoc testing revealed that the easier tasks with Id's equal to 4 and 5 did not produce significantly 
different means. However, the most difficult task (Id=6) was found to yield task means that were significantly 
different from the other two. The mean task time for Id=4 was 1.89 seconds, for Id=5 it was 2.15 seconds, and for 
Id=6 it was 2.74 seconds. Figure 4 graphically displays the effects of task difficulty on direct viewing performance 
Both a linear fit, y=0.138+0.424x, and an exponential fit, y=0.882* 10^(0.08 lx), were performed. As was found with 
the video viewing results, the increases in mean task time displayed more of an exponential tendency than a Unear 
one. 



0123456789 
INDEX OF DIFFICULTY (BITS7TASK) 

Figure 4 - Task Difficulty Effects on Direct Viewing Performance 
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Effects of Frame Rate 

Frame rate made a significant difference in performance (table 1). The Newman-Keuls post-hoc test showed 
that all three frame rates were significantly different. Mean task time with a frame rate of three frames per second 
(fps) was 5.36 seconds, for five fps it was 4.48 seconds, and for 30 fps it was 2.56 seconds. These results are shown 

graphically in figure 5. 



Figure 5 - Frame Rate Effects on Video Viewing Performance 


Interaction Effects of Frame Rate with Force Feedback 

The interaction of frame rate and force feedback was also noticed to make a significant difference in 
performance times (table 1). Post-hoc testing results showed that three frames per second (fps) with force feedback, 
five fps with force feedback, and thirty fps without force feedback were found to produce mean task times that were 
not significandy different from each other. Three fps without force feedback and five fps without force feedback were 
not significantly different from each other. Thirty fps with force feedback was significantly different from all other 
conditions. These results are graphed in figure 6. 

While at each frame rate, force feedback made a significant improvement in performance times, force 
feedback yielded a larger performance improvement at lower frame rates than at higher frame rates. Even at 3 fps, 
force feedback provided such a large improvement in performance that the mean task time was not significantly 
different from 30 fps without force feedback. 



Figure 6 - Interaction of Frame Rate and Force Feedback 
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Effects of Subtended Visual Angle 


Subtended visual angle did cause significant differences in mean task times (table 2). The Newman-Keuls 
jxwt-hoc analysis determined that the three subtended visual angles produced task time means that were significantly 
different from each other. The mean task time for a subtended visual angle of 3.28 degrees was 1.92 seconds, for 1 .64 
degrees the mean task time was 2.27 seconds, for 1.09 degrees the mean task time was 2.59 seconds. These results 
are displayed in figure 7. 



Interaction of Subtended Visual Angle with Force Feedhack 

The interaction of subtended visual angle with force feedback was found to make a significant difference in 
perfomance times (table 2). The Newman-Keuls post-hoc tests revealed that at all three subtended visual angles 
force feedback versus no force feedback made a significant difference. 

While operating with force feedback, decreasing the subtended visual angle did not significantly increase task 
times. Further, force feedback was able to improve performance at the smallest subtended visual angle by a margin 
large enough to make the task times not significantly different from those observed for the largest subtended visual 
angle without force feedback. Performing tasks without force feedback, and at a 3.28 degree subtended visual angle 
was not significantly different from that at 1.64 degrees without force feedback. However, performance at 1 09 
degrees without force feedback was significantly different from that for the other two subtended visual angles without 
force feedback. These results are displayed in figure 8. 



SUBTENDED VISUAL ANGLE (DEGREES) 

Figure 8 - Interaction of Subtended Visual Angle with Force Feedback 
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Effects of View ('Video Viewing at 30 f n s vs. Direct Viewing at 1 M degrees) 


When frame rate and subtended visual angle were similar, video and direct viewing mean task times were not 
found to be significantly different. This indicated that for these experiments whether the view was direct or video by 
itself did not make a difference and that the primary visual variables affecting performance were subtended visual angle 

and frame rate. 


6. Conclusions and Recommendations 


When using a remote manipulator system without force feedback, like the space shuttle remote manipulator 
system (RMS), and with direct vision, a relatively large subtended visual angle should be provided to the operator i 
possible. The experimental results suggested that in direct viewing telemanupulation, an adequate subtended visual 
angle could compensate for performance degradations that were due to the operator being at large distances from the 
task board. A larger subtended visual angle could be yielded by providing a larger target to increase the size o th 
image on the operator's retina thus increasing the subtended visual angle. Additionally, since force feedback would 
not be present, one would also expect that performance will be degraded more at smaller angles than it would be if 
force feedback were present 


The experiments indicated that frame rate and subtended visual angle were the visual feedback variables that 
affected performance significantly in the experiments, not whether direct or video viewing was implemented. Some 
previous studies (performed without a transmission time delay) concluded that position and force feedback are 
reconstiuctible to a large degree in teleoperation, but recreating the visual image as feedback to the operator was more 
challenging due to the loss of information (such as stereoscopic vision) when viewing a television monitor 119]. Uur 
results suggested that there was not as a significant decrease in performance due to the loss of information when 
going from direct viewing to video viewing as might be expected. This leads one to conclude that if a manipulator 
without force feedback were being used (such as the RMS), and direct vision yielded a small subtended visual angle, it 
would be wise to use video transmission to provide a larger subtended visual angle. Although the video monitor may 
not provide stereoscopic vision, performance would probably be improved with the larger subtended visual angle. 
Thus if a choice is given between direct viewing with a small subtended visual angle against video viewing with a 
high frame rate and larger subtended visual angle, video viewing could be the wiser choice. 


However, there may be other explanations for view not having a significant effect on mean task time. For 
example, the effects of stereo vision could be greater at shorter distances than at longer distances. Therefore it is 
possible that the eight foot distance was too great to utilize the full advantages of stereo vision. This is a topic or 
future research. Nevertheless the results found in these experiments suggest that the view itself did not have a 
significant effect on mean task times. 


If teleoperation were to be controlled from a ground control station or a space station workstation with a 
video monitor (such as with the control of an orbital maneuvering vehicle or flight telerobotic servicer), operating at 
very low frame rates below 3 fps should be avoided unless large performance degradations are acceptable. The 
reduction in mean task times was found to occur at a faster rate when going from 5 fps to 3 fps than when going 
from 30 fps to 5 fps. This suggests that frame rates between 5 fps and 30 fps may produce performance results that 
would more likely meet acceptable performance criteria than frame rates below 5 fps. Thus if there is limited 
bandwidth available for frame rate and depending on the task, reducing frame rate from 30 fps may be acceptable until 
a cutoff frame rate is reached beyond which performance would be below the accepted level. It may be possible to 
reduce frame rate to a larger degree without harming performance by a great margin if force feedback is present. If 
force feedback is not present, it would probably be important to have' the video transmission at a high frame rale. 


Force feedback was found to make up for many of the performance degradations due to decreased feedback in 
the visual feedback channel. Force feedback significantly improved performance at all frame rates, and was 
particularly helpful at the lower frame rates. In direct viewing performance, force feedback was found to have a 
stabilizing effect. When subtended visual angle was decreased with force feedback, there was not a significant increase 
in task limes. Additionally, force feedback yielded a larger improvement in performance time over the no force 
feedback case as subtended visual angle was decreased. Therefore whenever visual feedback conditions are extremely 
poor and cannot be improved, the use of force feedback could very well improve performance times to acceptable 
levels. 
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The results also suggest that force sensing was probably more important than vision for the insertion of the 
block into slot. Some scenarios may dictate that force feedback is impossible or undesirable, such as when used with 
a transmission time delay. But if force feedback is available and the task and environment do not prohibit its use 
force feedback should be utilized. 

As tasks become increasingly difficult, designers should not assume linear increases in task times. Task 
times can increase at increasing rates. This also suggests that beyond certain difficulty measures, performance time 
can increase beyond acceptable ranges. 

For any manipulation task the effects of the different feedback variables will be unique, making broad 
generalizations ill advised. However the conclusions presented here indicate that task difficulty, force feedback, and 
the visual feedback parameters of frame rale and distance (or subtended visual angle) all can have significant effects on 
human performance for telemanipulation. 
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